Technical Field
The present disclosure generally relates to an image processing apparatus, image processing method, and computer-readable medium.
Description of the Related Art
In extracting an image of an object included in an image, a method called background subtraction may be known widely and used commonly. This method may compare an input image (hereinafter the input image will mean an image resulting from imaging an object) with a background image which is an image captured without including the foreground (hereinafter the image captured without including the foreground will mean an image of an object in the input image) of the input image and extract an area which has changed from the background image by determining the area as being a foreground, where the input image and background image are prepared separately.
In the background subtraction method, a change in the background itself caused by appearance of shadows may be extracted by being determined to be a foreground, resulting in low extraction accuracy.
In some aspects, there may be a method that distinguishes a shadow area by analyzing an area including extracted object and shadows. In some aspects, there may be a method that extracts a foreground excluding a foreground-like shadow area by unifying a change assessment of luminosity having a somewhat wide tolerance and a change assessment of color tones (e.g., hue and saturation, color difference, or the like) having a narrow tolerance. This method may assume a monochromatic lighting environment and it is claimed that changes in the background due to shadows occur only in luminosity (luminance direction component).
For example, HSV (Hue, Saturation, Value) color space may be used to process brightness (luminosity) and color tones (hue and saturation) separately.
In some aspects, there may be a method for comparing only color tones out of chromaticity by normalizing values of each of RGB (Red, Green, Blue) channels (hereinafter the chromaticity will mean a relative value resulting from removing brightness from RGB values or the like, where the RGB values are those commonly used to display colors on screens of computers and the like).
In some aspects, these methods described above may be applicable when the color of shadows does not change from the color of the background under monochromatic lighting. In other aspects, the methods may not be applicable when the color of shadows changes to a color different from the color of the background under polychromatic lighting.
In some aspects, there may be a method for learning a relationship between changes in illumination intensity of direct light and changes in pixel values in a dichroic lighting environment made up of direct light (Id) and diffused light (Is), assuming that shadows are produced when direct light is blocked.
Lighting components in a background and shadow area in a dichroic lighting environment are illustrated in FIG. 14. The changes in the illumination intensity of the direct light may be controlled by a weighting factor α (0≦α≦1.0). At this point, in the same background, lighting components acting on color information in a shadow-free state (background) and shadowed state (shadow) may be represented by Id+Is and αId+Is, respectively (hereinafter the color information will mean quantified color values represented by RGB, YUV, or HSV, where YUV represents luminance, blue color difference, and color difference from red). FIG. 15 is an explanatory diagram illustrating the above relationship in the RGB color space. The solid line represents the relationship between a shadow-free state and shadowed state obtained by learning, and any object existing on or near the solid line in the background is determined to be a shadow. An input pixel value distant from the solid line may be distinguished as belonging to a foreground.
In some aspects, there may be methods for calculating spectra using sunlight models.
Since some methods assume monochromatic lighting, shadows may not be distinguished properly when the color of shadows changes from the color of the background under the influence of other lighting.
In some aspects, some methods may not be applicable to an environment in which the color changes in an outdoor environment as in the case of sunlight, i.e., to a case in which the spectra of direct light and diffused light change. As an example, FIG. 16 illustrates an explanatory diagram of a case in which color information about direct light and diffused light differs from that existing during learning. A relationship learned in a certain lighting environment is indicated by a solid line. When the hues and saturation of direct light and diffused light change with time as in the case of sunlight, the relationship between the background and shadow in terms of color information exhibits a characteristic (indicated by a broken line here) different from that of the solid line existing at the time of learning. In this case, the shadow in this environment may not be distinguished properly unless the relationship indicated by the broken line is not learned anew. That is, this method may not be applicable to the outdoor environment in which the spectra of the direct light and diffused light of sunlight change from moment to moment.
In learning the relationship between changes in the illumination intensity of direct light and changes in pixel values, it may be often the case in actual practice that a state which is a shadowed state is learned erroneously as a shadow-free state. The relationship between background and shadow based on such erroneous learning data may take on a form such as illustrated in FIG. 17. In this state, when an input image captured in a shadow-free sunny place is used, it may not be possible to distinguish whether a portion making up the input image is a foreground or a shadow. This may be because with some methods shadow determination accuracy deteriorates due to mislearning. To improve the determination accuracy of this method, learning sets may be increased. In some aspects, a great deal of cost may be required for computation processes, and the method may be difficult for practical use.